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ABSTRACT 



This report describes the performance of New York City's Job 
Training Partnership Act (JTPA) adult training providers. It discusses 
challenges currently faced by providers, and recommends strategies for 
improving the performance of the city's employment and training system, 
including those arising from the implementation of the Workforce Investment 
Act. Data for the evaluation of the providers were gathered through visits 
to, and records from, 53 training providers serving about 5,300 participants, 
and from 6 organizations running testing and placement centers serving 
another 8,000 participants. The evaluation found that, although many 
providers met their contractual standards, their performance varied widely, 
with placement rates ranging from 50 percent to more than 83 percent. Most 
contractors met their wage-placement goals. Challenges facing the programs 
include staff recruitment and burnout, communication between providers and 
New York City's Department of Employment, performance-based contracting, 
performance standards, collecting performance data, lack of development of 
training programs for new industries, and conflicts with other programs. 
Recommendations for program improvement included the following nine changes: 
(1) aligning local performance standards with national goals; (2) gathering 
timely information; (3) weeding out poor performers; (4) providing staff 
development; (5) creating a loan fund to meet short-term cash needs; (6) 
analyzing local labor market trends; (7) offering multiple points of entry; 

(8) expanding postplacement services; and (9) collaboration between these 
providers and one-stop operators. (KC) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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Introduction 



This report: 



In late 1998, the New York City Department 
of Employment (DOE) decided to develop 
a technical assistance plan for its contracted 
employment and training providers. At the 
time, DOE was about to move forward with 
a new performance-based contracting sys- , 
tern and wanted to know what assistance 
would be most useful to providers. The 
Department’s expectation was that a new 
technical assistance initiative would help 
providers through the transition to the new 
system, as well as improve the overall perfor- 
mance of providers, which in turn would 
benefit participants. To design the plan, 
P/PV and DOE agreed to examine hot only 
contractors’ immediate concerns, but to 
also explore challenges related to the imple- 
mentation of the newly enacted Workforce 
Investment Act (WIA). 

In February 1999, P/PV began an assess- 
ment of the contracted providers that 
sought to understand their concerns and 
elicit their suggestions for technical assis- 
tance. The assessment comprised two 
strategies. First, P/PV administered a survey 
to all Job Training Partnership Act (JTPA) 
Title I LA (disadvantaged adults) and Title 
III (dislocated workei's) contractors to get 
an overall picture of the field. Second, it 
made site visits to several contractors, 
representing a cross-section of providers. 

In addition, P/PV held several meetings 
with DOE staff throughout the course of 
the project. 



• Describes the performance of New York 
City’s JTPA adult training providers; 

• Discusses challenges currently faced by 
providers; and 

• Recommends strategies for improving the 
performance of New York City’s employment 
and training system, including those arising 
from the implementation of the Workforce 
Investment Act. 

During the course of this assessment, the 
employment and training landscape shifted 
dramatically with responsibility for the Tide 
IIA programs moving from DOE to the 
Human Resources Administration (HRA) in 
late spring. This shift raises important chal- 
lenges and opportunities for the future of 
workforce development in the city; though 
the timing of this change precludes P/PV 
from addressing them in this report. 
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New York City's Job 
Training Performance 



Within the New York CityJTPA system, a 
variety of programs serve adults and youth. 
This report focuses on disadvantaged adults 
(Title IIA) and dislocated workers (Title III). 
In the 1997-98 program year, 53 training 
providers served about 5,300 participants 
with $27 million in public resources. In 
addition, six organizations ran Testing, 
Assessment and Placement Centers, known 
as TAP Centers, serving another 8,000 par- 
ticipants with approximately $9 million.* 

To understand the issues facing DOE as well 
as local contractors, it is important to put 
the city’s overall performance in some con- 
text. Across New York state, the 33 Service 
Delivery Ai eas- (SDAs) are judged accord- 
ing to a series of standards established by 
die U.S. Department of Labor (USDOL). 
Individual SDAs are measured against six 
core standards: adult follow-up employment 
rate'^ and follow-up weekly earnings;^ welfare 
recipients’ follow-up employment rate and 
follow-up weekly earnings; and youth entered 
employment and employability enhance- 
ment rates. As Table 1 indicates. New York 
state’s outcomes exceeded die federal stan- 
dards in 1997.^ 



The performance standards for individual 
SDAs are set by USDOL, but can be adjust- 
ed by governors to reflect local conditions. 
The USDOL standards take into considera- 
don factors relating to both the participants 
and local economy, including the percent- 
age of participants who are female, 
minorities, not high school graduates and 
not in the labor force, combined with the 
local unemployment rates and percentage 
of families with incomes below the poverty- 
leyel, among others. In addition, governors 
are allowed to adjust performance levels 
based on additional factors such as serving 
other hard-to-serve populations, the type of 
services being provided, and regional varia- 
dons in local economic conditions. USDOL 
creates “tolerance ranges” for how much 
governors can adjust the standards. 



Table 1 : New York State JTPA Performance, Prograrh Year 1997 







Standards 


Outcomes 


Adult 


Follow-up Employment Rate 
Follow-up Weekly Earnings 


56.2% 

$296 


■ 74.6% 
$312 


Welfare Recipients 


Follow-up Employment Rate 
Follow-up Weekly Earnings 


46.6% 

$252 


72.5% 

$283 


Youth 


Entered Employment Rate 
Employability Enhancement Plate • 


39.3% ■' 
35.6% 


56.0% 

55.0% 



O 



Source: Data provided by the New York City Department of Employment. 



-Wliile the state has performed well, New - Chart 1: Wide Disparity in Placement Rates 
York City has not been as successful. As Table of Providers 

2 demonsD*ates, New York City failed to meet 
USDOL standards in three categones and 
just met the governor’s adjusted standards. 

(As a point of comparison, the Yonkers SDA 
was the only New York SDA with lower per- 
formance goals than New York City’s, with 
its adult follow-up employment standard set 
at 37.5 percent; nevertheless it achieved a 
follow-up employment rate, of 74.6 percent.) 

Why is New York City having a difficult time 
reaching its performance standards? 

While there is overlap between the perfor- . 
mance measures that the overall SDA must 
meet and the' performance measures performance levels of individual Title IIA 

. incorporated into individual contracts for contractors for the first time using automat- 
providers, specifically regarding measures , ed data from the Standardized Program 
for retention, there are also many inconsis- Information Reporting (SPIR) system, 
tencies among the .two sets of performance . Based on contractors’ performance data 
standards. For example, DOE performance from 1996, approximately 50 percent of 
evaluations place the most emphasis contractors exceeded the contractual goal 

(points), on training-related: placements of 65 percent. However, as indicated in 

and employment at termination, neither of Chart 1, there is \\dde disparity in the place- 

which are taken into consideration in the ment rates of providers with the bottom 

national performance standards. Therefore, quartile of providers achieving a median 
contractors could fail to meet the federal ’ placement rate of 50.1 percent while the . 
performance objectives under JTPA, but top quartile achieved a 82.7 percent medi-. 
still meet the m^yority of the performance an placement rate. The median wage at 
measures in their contract. placement was $9.01 per hour, with a high 

' . of $11.85 and a low of $6.97, and all but 

Ironically, most individual providers perform two contractors achieved the contractual, 
fairly well when measured on their contractu- goal of $7.00 per hour, 
al standards.^’ In 1998, DOE examined the 

Table 2: New York City JTPA Performance, Program Year 1997 



2 40 



30 





Bottom Quartile 2nd Quartile 3rd Quartile Top Quartile. 







USDOL 

Standards, 


Governor- 

Adjusted 

Standards^ 


Outcomes 


Adult 


Follow-up Employment Rate 
Follow-up Weekly Earnings . 


.43.2% 

$330 


40.0% 
$318 ■ 


43.0% 

$325 


Welfare Recipients 


■ Follow-up Employment Rate 
Follow-up. Weekly Earnings 


33.7% 

$291 


29.5% 

^$278 


. 37.9% 
$310 


Youth 


Entered Employment Rate 
Employability Enhancement Rate 


41.5% 

43.6% 


36.5% ■ 
38.9% 


39.6% 

45.2% 



^ Source: Data provided by New York City. Department of Employment. 
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The major factor in New York City’s difficul- 

• ty in reaching its performance standards is 
the substantial impact of tlie Testing, 
Assessment and Placement (TAP) Centers 
on key outcomes. The TAP Center model 
operates differently from other JTPA-fund- 
ed programs, with, as tlie title indicates, 
more emphasis on assessment, short-term 
pre-employment training and placement. In 
addition; the TAP Centers operate at a 
rtiuch higher volume than do JTPA training 

• providers, with six organizations contracted 
to serve over 8,000 participants each pro- 
gram year, compared to 5,300 served by 
long-term training providers. TAP Centers 
are far less expensive, however, spending 
roughly $1,125 per participant, compaied 
to training providers which spent about 
$5,000 per participant. 

A 1998 report from DOE found that during 
the 1996 progiam year, the TAP Centers 
consistently had lower placement rates than ’ > 
did the adult training providers: “No matter 
what occupation they were placed in, or 
trained and placed in, the TAP Centers’ 
placement rate was lower than the training 
programs.” The report also examined the 
effect of education, labor force status and 
ethnicity on placement rates, and conclud- 
ed, “even if the TAP centers served only the 
most advantaged, the placement rate would 
still be below the training programs rate” 

(Dei et al., 1998, p.l4). 



Although the performance data in Table 3 
reflect performance at placement, not at 
follow-up 90'days later, it is clear that long- 
term training providers do achieve higher 
performance levels than do TAP Centers. It 
appears likely that if the city ends support 
of the TAP Centers, as is currently planned, 
it will be able to meet the USDOL perfor- 
mance standards. 

Because DOE intends to phase out its con- 
tracts with TAP Ceiiters, P/PV focused its~ 
analysis and recommendations on the issues 
facing long-term training providers. DOE 
and P/PV also agreed to limit the scope of 
■the assessment to contractors serving adults, 
since they will be most affected by the 
Workforce Investment Act. 



Table 3: TAP Compared to Title II Adult Training, Program. Year 1996 



Percent Placed Wage at Placement 



All , 


Welfare 


All . 


Welfare 




Recipients 




Recipients 



TAP Adult 


41-6% 


29.7% 


$9.22 


$7.95 


Adult Training 


• 60.9 . 


55.1 


9.09 


8:95 



Contractors 

Source: Dei et al., .1998. 




8 



5 



Major Challenges Facing New York City's 
Employment Providers 




EKiring the past several years, there have 
been a number of significant changes in the 
emplo)anent and training field in New. York 
City. From the higher performance stan- 
dards for service providers, to movement, 
toward a centralized assessment and referrals 
system, and the implementation of perfor- 
mance-based contracting for dislocated 
workers, change has. been occurring at a 
rapid pace. With these changes, providers 
have been faced with many challenges, from 
continuing struggles with staff development 
arid technological changes to concerns about 
the impact of welfare reforrn. Providers con- 
sistently remarked on these tumultuous 
times. On top of the day-to-day struggles, 
providers are facing challenges posed by tlie 
Workforce Investment Act. Even’ providers 
who have been working in the field for . , 
decades said nothing has prepared them for 
• what.they will face over the next year., 

What does the field of employment 
providers look like?, , 

From contract data provided by DOE 
(Table 4), median enrollment of Title IIA 
contractors is 90 participants; with a medi- 
an budget of approximately $476,000 per 
year. For Title III providers, the median 
erirollment is 38, witli a budget of $240,000 
per year. Curreritly, 53 different contracted 
organizations provide employment training' 



services. While some providers run multi- 
ple JTPA programs, the vast m^ority 
(74%) run juk one type of program. 

To deepen our understanding of the issues- 
facing the field, P/PV conducted a survey 
of all current (PY 98) Title IIA and Title III 
DOE contractors. We achieved a response 
rate of 70 percent, with 37 of the 53 con- 
tractors responding.'-^ The survey was ’ 
designed to provide a portrait of the 
emplo)Tnent and training community in 
New York City. Overall, the field of employ- 
ment and training providers in New York 
City can be described as a community of ' 
veterans: 75 percent of survey respondents, 
have, been in the job training field for 1 0 
or more years, witli 69 percent having been 
contractors with DOE for seven or more 
years. The vast majority of providers are 
nonprofits (78% of respondents). However, 
there is also a “big guy, little guy’’ phenome- 
non in New York City; 62 percent of survey 
respondents, serve between 50 and 250 pa'r- 
. ticipants a year; 22 percent of providers ' 
serve over 1,000 participants each year. 




Table 4; Characteristics of Contractors, Prograrn Year 1998 



Title IIA 
Title ill 



Number of Contractors^ 

39‘, 

25 



Median Enrollment 

90 . \ 

38 



Median Contracted Budget 

. $476,000' - 

240,000 



Source: P/PV analysis' of DOE contract data. 
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Areas of Current Concern 



Staff 

While there are significant policy and 
admihistratiye changes occurring in New 
York City’s employment arena, providers 
cited staff recruitment, development and 
retention as their most important challenges. . 
The vast m^yority of providers (81%) have 
staffs of 15 people or less; 44 percent have 
one to five people. Twenty-two of the 37 
survey respondents cited staff development 
' as a challenge or an area of their program 
that needs strengthening. 

Providers discussed the high incidence of 
burnout, due at least pardy to the pressure to 
meet performance goals. Directors and staff 
often talked about “meeting the numbers” 
and being “hammered by our placement 
goals” as a m^yor source of tension as well as 
a driving force in the organization.' 

Some providers contended that as a result 
of this pressure, organizations are having a 
difficult time retaining staff. Half the pro- 
grams we visited had recent significant staff 
turnover. Others mentioned the “lure of 
the private sector,” particularly in. the area 
of job development. Some providers voiced 
concerns about the effect of turnover on 
their performance measures. As one progiam 
director commented, “losing one good Job 
developer can affect performance numbers 
for two or three cycles until someone else is 
brought in and can rebuild relationships 
with employers.” 

While pressure to meet performance num- 
bers is one reason for staff turnover, the 
wages and benefit levels offered by providers 
could also be a reason, as well as the lack of 
career ladders within organizations. As one 
executive director noted, “I can’t have three 
program directors, can I? As staff gain expe- 
rience, tliey look for senior positions in 
other organizations.” 

Providers requested general staff develop- 
ment assistance in the areas of technolog)^ 
program development and communication 
skills. In addition, providers listed specific 

o 




staff responsibilities that require assistance, 
including marketing and recruitment, job 
development, placement and postplace- 
ment retention. 

It should be noted that in the responses to 
what technical assistance contractors found, 
useful, several cited DOE/PIC-sponsored 
staff training, specifically in the areas of job 
development and employer relations. From 
the in-depth interviews, it appears that train- 
ing benefits its intended audience; in general 
those who could be considered weaker per- 
formers rated the sessions very beneficial, 
while those providers who are better per- 
formers did not find the sessions as useful. 

Communication between providers and New 
York City's Department of Employment 

Many of the providers are concerned about 
a trend toward less communication between 
the providers and DOE. All nine organiza- 
tions we visited said it was very difficult to 
start and maintain a dialogue with the 
agency, outside of bidders conferences and 
contract negotiations. 

Several providers recalled a time when the 
agency dispatched field staff to visit conuac- 
tors outside of program evaluations and 
conu'act renewals. Providers said it was dur- 
ing these visits that the agency staff got to 
know its providers and understood the issues 
they were facing. Thus, during contract 
negotiations providers and the agency were 
more likely to be on the same page about 
the issues. The lack of direct contact results 
in a lack of information about impending 
changes. For example, all of the information 
contractors have received about tlie possible 
changes under WIA came from sources out- 
side of the Department. 

Lack of flexible funds/cash reserves 

Providei's acknowledged the importance of 
flexible funds to fill cash flow gaps due to 
slow payment of city contracts, or to provide 
start-up funds for operation under a perfor- 
mance-based budgeting system. 
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Chart 2: Almost Half of the Providers Have - 
No Cash Reserves 




-I Have Cash Reserves 
□ Do Not Have Cash Reserves 

Source: P/PV analysis of DOE contract data. 

As Chart 2 demonstrates, nearly half the 
survey respondents have no cash reserves. 
When asking one fiscal director if the 
organization had cash reserves, she said, 
“Yeah, [the Executive Director's] college' 
fund for her kids.” Of the 19 organizations 
that do have cash reserves, there is wide 
variance in the amount, ranging from as 
little as $50,000 to as much as $2 million, 
with a median of $325,000. Half the cash 
' reserves covered less than 25 percent of the 
organization’s job training budget. 

This lack of flexible funds is of even greater 
concern^ given providers’ reliance on public 
’ sector funding. Of the survey' respondents, 
nearly half relied on DOE funds for 70 per- 
cent or more of their job training budget; 
and DOE was the only, source of support 
for 29 percent of providers. Half of the 
respondents also received other New York 
state contracts. Only eight respondents* , 
received philanthropic support, represent- 
ing a median of 12 percent of their budget, 
and tuition and fees for training for : 
employers were sources of funds for only 
four providers (two each). > . : 
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Performance-based contracting 

Wlien asked about performance-based con- . 
tractin^, many conu actors said they would 
have welcomed the shift, under certain con- 
ditions. Some Title III conuactors, who had 
just moved to performance-based contracts, 
commented, it was not a “real performance- 
based system,” The system allows for 
contractors to be penalized if they do not 
meet their performance goals. However, 
there is no reward if they surpass their goals. 

Without exception, the primary budget 
concern of providers was the integration. of 
line-item and performance-based budgets. 
Currently, budgets are performance-based 
with benchmarks, throughout the program. 
However, contractors are also obligated to 
provide line-item back-up for all of their 
expenditures. This “double bookkeeping” 
requires tremendous effort bn behalf of 
.. the providers. ' 

Performance standards 

While most providers were able to meet the 
performance standards in their contracts, 
Title HI providers, in particula.r, were con- 
cerned about the latest increases in standards 
and changes in the definitions of standards. 

Title III contmctors recently went through a 
new round of contract negotiations that set 
die placement wage at $11 per hour. Some 
believed the wage too high, considering the 
low..skill level of many of the participants. 
However, of most concern to providers was , 
the shift from an average wage at placement 
to a stipulated wage for all placements. Many 
programs that would have no problem meet- 
ing the average wage of $11 per hour, with 
low and high wages averaging out to die 
standard, will have a difficult dme meedng 
die new objecdves. Furthermore, USDOL. 
and DOE define placements as working at 
least 20 hours per week. As one director stat- 
. ed, “DOE doesn’t care if they [pardcipants] 
are making $10.99 [an hour] in a 40-hour-a- 
week job, to them it just doesn’t count.” 
Contractors believe participants would be 
better off with full-time jobs even if hourly ■ 
wages are slightly lower. 

1,1 . ' 



The challenges of these new standards were 
particularly evident in a training program 
that has stiong ties to employers. Many pro- 
gram graduates enter into apprenticeship 
programs upon completion, earning $8 to 
$10 per hour. However, within the first year 
of an apprenticeship, participants could eas- 
ily be, making $12 to $15 per hour. Under 
the new regulations, this provider would not 
be able to meet its performance standards. 
Asked what they would do, the director said, 
“Well we thought long and hard about it. 

. We finally signed the new contract. Our first 
class will graduate in June, and we will just 
have to see what happens. But I will not dis- 
courage any of iny trainees not to take a job 
because it doesn’t meet the performance 
goal. We’ll just deal witli that when it comes.” 



Technology and performance data 

Although providers have been fairly sue- , 
cessful in meeting performance standards, 
collecting performance data has proved 
much more difficult. Currently, providers 
use the Automated Case Management 
System (ACMS) to collect and transmit per- 
formance data to the Deparunent. 

While providers commented on technical 
difficulties with the system, the primary 
otyection was the time commitment required 
from staff. Data entry tasks were often spread 
out among different staff, often leading to 
errors and delays in submitting data. This in 
turn led to incomplete performance data 



Chart 3: Targeted Industries for Job Training 



Number of Programs 0 



5 



Customer Service 
Health Care 
Clerical 
Computer 
Hospitality 
Building Maintenance 
Accounting 
Automotive 
Electrical Tech. 
Human Services/Counseling 
Transportation 
Construction/Building Trades 
Banking/Financial Services 
Food Service 
Other 
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(Respondents were able to offer more than one response) 
Source :P/PV analysis of survey data. 
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from providei's and required additional staff 
time to rectify errors. Some providers want- 
ed to have dedicated staff for the data entry 
work, but DOE resources did not support a 
stiff position. 

One critical issue contractors face is the dif- 
ficulty in manipulating the data within the 
ACMS for their own purposes. Much of the 
data recorded is important for contractors’ 
program management; but the system has' 
limited capabilities in producing reports for 
providers. Few organizations appear to use 
other computer systems to collect and 
manage program information that would . 
be useful for future planning. At best, staff 
had simple tables or spreadsheets detailing 
performance information; most just had a^ 
myriad of paper files that would have to be 
examined individually to get an overall 
picture of the agency. This. may reflect, in 
part, the fact that smaller organizations are 
unable to have people on staff with the. nec- 
essary computer expertise. In fact, it was the 
smaller organizations responding to the 
survey that rated the technical assistance by 
DOE regarding the ACMS as most useful. 



Industry sectors and labor market information 

Many contractors have been training for 
and placing participants in the same indus- 
tries for many years (Chart 3). Among the 
organizadons we interviewed, all had 
trained for the same industries since they 
began running training programs, making 
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minor adjustments to keep up with the 
changes in the field, siich as upgrading soft- 
ware. In addition, only one provider had 
any component that could be considered 
customized training for an employer. A few 
providers discussed plans to expand their 
programs to include customer-service train-, 
ing. While that is an occupation in demand, 
some were concerned about the low pay ' 
and prevalence of part-time work. 

Part of the lack in the development of 
training programs for new industries has 
been the limited access to, or familiarity 
with, labor market information. Several 
providers knew information was generated 
at either the state pr local level, but did not 
know how to access or utilize it to make 
. informed decisions about their programs. ' 

To promote stronger ties with employers,. , 
DOE requires all providers to have a Private . 
Sector Advisory Committee (PSAC) , to 
provide guidance to contractors on their 
programs and recommendations for 
changes. The PSACs of the organizations 
yisited meet a few times a year, but do not 
appear.to be a major influence oh the 
organization. Some providers’ “links” to ’ 
employers were simply periodic phone calls 
to employers with whom they had estab- 
lished good relationships or a perusal of the 
want ads. However, the organizations that 
had employers on their boards seemed to 
have the strongest attachment to the private 
sector, both in terms of increased placement - 
‘ within these companies as well as contem-- 
plating rnore changes in their programs. 

Conflict with Work Experience Program (WEP) 

The vast m^yprity of providers cornmented 
on the negative influence the city’s Work 
Experience Program (WEP) has had on 
public assistance recipients’ ability to partic- 
ipate in training. WEP has a strong Work 
First element^ requiring public assistance < 
recipients to conduct a job search for four 
to six weeks. If employment is not found, 
then participants are enrolled in a Work . 
Experience slot; working 21 hours per week. 



The biggest challenge posed for providei% 
has been the removal of participants from 
training programs to fulfill WEP assign- 
ments. Providers are concerned that this 
will have an adverse effect on their perfor- 
mance measures because participants are 
being pulled in the middle of their training. 
Interviewed providers said usually two to 
three trainees are pulled every cycle.. 

Overall, providers have seen a marked drop 
in the number of public assistance recipi- 
ents in their programs,- which they attribute 
to the WEP program. ' . 

One of the greatest divisions within the . 
community of employment and training 
providers is their interaction with the Work 
Experience Program. Providers deal with 
the program in a variety of ways. Some 
providers work closely with HRA Job 
Centers to have the internship or out-place- 
ment components of their programs qualify • 
as a WEP assignment. Some programs even 
have the participants work part time in the 
organization to fulfill their WEP assign- . 
ment. However, these arrangements are 
coordinated individually with each case- 
worker,' so some participants are able to do 
this and others are not. , 

Some providers have decided not to get 
involved at all with the WEP program and 
have stepped up recruitment efforts for. 
noripublic assistance recipients: Others have 
^required public assistance recipients to 
obtain evening or weekend WEP assign- 
ments to fulfill their obligations. Regardless 
of providers’ views about working with the 
WEP program, they feel that public assis- 
tance recipients ai'e better off in training 
programs than in WEP assignments. 

Workforce Investment Act: Areas of Concern 

Just over the horizon is the. new Workforce 
Investment Act (WIA). Signed into law in 
1998, WIA sets out to fundamentally 
restructure the field of workforce develop- 
rnent. WIA attempts to sti eamline a myriad 
of federal training programs around the fol- 
lowing principles: 



'• Greater decision-making authority on the part .2. The role of cbminimity-based organizations. 



of states and localities through the creation of 
\ Workforce Investment Boards (WIBs); 

• Operation of a one-stop delivery system, which 
provides universal access to anyone in need of 
workforce development services; 

• Creation of a three-tiered service delivery. , 

. strategy: core,service available to everyone; 

..intensive services to those who are unable to 
' obtain employment on their own; and 
training for those who complete intensive 
services without finding employment; 

• Development of aiVindividual Training 
Account (ITA), or voucher, kystem as the 
method of payment to providers for training 
services; and 

• Increased accountability and customer. ^ 
choiceThrough enhanced tollection and 
dissemination of performance measures. 

At’ this stage, governance issues, particularly 
the appointment of members to the state 
and local WTBs, haye dorninated the discus- 
sion about WIA in New York. Many of the 
most crucial provisions of WIA will be 
determined by tliese boards, including 
selection of the one-stop operator, creation 
of eligibility criteria for service providers 
and development of the voucher system. . 
New -York has until April 1, 2000 to submit 
its plan to USD QL and is required to be^n 
implementation of WIA by July 1, 2000. 

Gene rally speaking, providers identified four 
broad concerns about the implementation 
of WIA in New York City: 

1. The role of training. Providers’ greatest ^ 
concern was about the de-emphasis on 
training and a strong focus oh Work First or, 
at most, short-term training and placement in 
the new system. The three-tiered service plan, 
the use of vouchers and one-stops appear to 
many providers 3S obstacles for'people who 
want to recdve training. / 
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. The majority of the survey respondents and 
, interviewees considered themselves CBOs and 
. could not see a clear foie for themselves in 
• WIA or in the planning process. One director 
; described the role the CBO plays in the 

community and said emphatically that people 
will not be attracted to a streariilined, one- - 
. , stop system when they are accustomed to 

dealing with their local CBOs. As one /director 
stated, “I don’t think people wake up one day 
“and say I want to go get some training. They . 
usually have some idea of where they want to ‘ 
^ ' go because they. have heard of us from family , 
or friends or because w'e have a presence in 
,the community.” r .. ' • 

3. The impact of universal access. Several 

^ providers are concerned about the move to 
universal access and what that would mean for 
the low-income population they traditionally 
serve. JTPA is Just one of several programs 
that are being merged under WIA, and some 
providers feel that the dominating presence of 

- programs like Wagner-Peyser, used primarily: 

by middle-class individuals, will push the. ■ • 

needs of low-incohiejob seekers aside. At the ^ 
other extreme, some providers are concerned 

^ that they may not be prepared to serve the, 
new populations included in the system, such 
as disabled and older individuals. ' ^ ^ 

4. Implementation timetable. Finally, the 
overwhelming concern among providers is the 

■ delay in planning for the implementation of 
WIA, V\Tiile states have until July 1, 2000 to 
implement all the provisions of the new act, ‘ 
little planning appears to have taken place at : 

- either the state or local level in New. York. ■ 
Providers have spent considerable time and 
effort trying to anticipate the impact of WTA 

. and are concerned that delays in planning 
will lead to delays in implementation,, causing 
' financial. difficulties for their organizations'. 

In addition to these general concerns, . 
providers discussed a variety of concerns- 
regarding the specific provisions of the 
‘Workforce Investment Act. 
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Creation of a one-stop system 

One of the m^yor structural changes under 
WIA is the creation of a one-stop center as 
the first point of contact for anyone seeking 
workforce development services. While 
many of the details are still being developed, 
it is within these one-stops that interested 
participants will be tested, assessed and a 
determination made as to their eligibility for 
training services. For many localities, this is a 
stark departure from current practice, par- 
ticularly in New York City. 

. Currently in New York City, JTPA-contr act- 
ed providers are permitted to conduct their 
own eligibility determination and thus have 
control over who is admitted, based on 
JTPA guidelines and their own criteria. To 
recruit potential participants, contractors 
have employed a variety of mechanisms 
including direct recruitment, flyers, news- 
paper advertisements and word of mouth. 

While this has been the basic model, two 
important initiatives have been operating 
in New York to move the city toward a cen- 
tralized systern of assessments and referrals. 
Beginning in 1989, the first Worker Career 
Center (WCC) opened in Manhattan, even- 
tually expanding to all five boroughs to serve 
dislocated workers. The Centers provide 
basic readjustment assistance to dislocated 
workers, such as resume preparation. Job 
listings and directjob placement, as well as 
testing, assessment and referrals for those 
seeking training services through JTPA. 
Currendy, the five WCCs'' are responsible for 
all referrals to Tide III providers across die 
city, serving approximately 10,000 to 12,000' 
people annually. , 

The Other inidative operates in the Bronx 
for Title IIA (disadvantaged adults) under 
a similar model. Started in 1994, the 

' Neighborhood Career Development 
Centers'^ (NCDCV conduct outreach, 
intake, assessment and referrals for a net- 
work of JTPA providers in the Bronx.’^ The 
goal of the pilot is to provide an dbjecdve 
assessment of people’s need for Job training 



services by an endty other than the training 
provider. The two NCDCs conduct a two-day 
assessment of individuals before determin- 
ing if they are eligible to be referred to 
training providers or other educational ser- 
vices provided by the NCDCs. 

Both of these initiatives provide some ^ 
important insights into the potential world 
of one-stop operations .in New York City: 
Providers who currently use either the WCC 
or the NCDC system cited several issues. 

Number and qualifications of referrals. Many 
providers noted initial problems in receiving 
an adequate number of referrals to meet 
their enrollment goals. Compounding the 
enrollment issue was the fact that WCCs arid 
NCDCs often sent participants who were not 
qualified for contractors’ programs. One 
provider estimated that between 45 and 65 
percent of referrals were unqualified because 
of low math and reading scores. , 

Conflicting performance goals. Some 
contractors discussed the problem of 
inconsistent — and sometimes competing — . 
performance goals between the 
WCCs/NCDCs and the training providers. 
Both WCCs and NCDCs are Judged on 
performance standards that include refer- 
rals to other training programs, but not 
on whether participants complete the. 
training programs. Besides these conflict- ’ 
ing goals, a few providers spoke of 
competing with the Centers on perfor- 
mance measures. Since the Centers offer 
additional services to participants, there 
have been some disputes between the 
Centers and providers as to who provided 
what service to the participant and then 
who gets to “claiih” the participant in their 
performance figures. Staff suggested . 
instrumenting “reverse' performance mea- 
sures,” in which providers and Centers are 
held accountable for performance mea- 
sures that meet the others’ needs, such as 
providers who send participants to the 
center for referral or centers who refer 
qualified participants back to providers. . 



Lost participants. Providers are also con- 
cerned that without control of the referral 
and recruitment process, people who, are 
truly interested and in need of training ser- 
vices will get lost in the system. “It won’t be 
a ‘one-stop’ system,” said one* program 
director. “Fii’st they will come to, us for ser- 
vices, then we will have to send them to the 
‘one-stop’ as their second stop. Then, 
maybe they will get sent on to a service 
provider.” As an example, one provider stat- 
ed that once they were required to send 
interested participants through a centml- 
ized testing and assessment process, they 
lost about 80 percent of, the people they 
originally recruited. However, these partici- 
pants may have been referred to more 
appropnate training. 

The use of vouchers 

Perhaps the most far reaching change 
under WIA is the utilization, of Individual 
Training Accounts (ITAs), or vouchers, for 
payment to service providers. This change 
brings with.it tlie loss of guaranteed rev^ 

. enue for service providers through JTPA 
contracts, except for some special popula- 
tions and customized training with 
employers! In addition, it forces providers 
to engage in a new level of marketing to 
attract participants, and their vouchers, to 
tlieir. training programs. - 

Since planning is still in its infancy, not sur- 
, prisingly, many contractors had questions 
about the mechanics of a voucher system: 
how much would they be worth; how would 
they be redeemed; what happens if a 
trainee drops out; and what is, the process 
to determine who is eligible for a voucher? 
However, two overriding issues will present 
challenges to providers regardless of the 
rhechanics of a voucher program. 

First, the sti ucture of many programs may 
need to be altered to offer multiple entry 
points: Of all eight programs visited, all but 
one had a set 20- to 22-week training sched- 
ule, where entry into the program occurred 
only twice a yean‘d Other jurisdictions that 
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have moved to a voucher system suggest 
having multiple entry points into the 
system so people can use their vouchers 
when they are issued, not when training ^ 
programs are recruiting. ' : 

Second, the financial characteristics of 
vouchers will pose a rnajor challenge for. 
many of the providers. Basically, a voucher 
systerh' pays training providers for each 
individual participant once s/he is enrolled 
in die progiam. Hence, diere are no start-up 
funds and no guarantee of a total amount — 
it. is all dependent oh the number of 
participants the organization enrolls. 
Therefore, providers will need to have ' 
flexible and diversified funding streams to 
survive under vouchers. . 

Many of the organizations visited were 
grappling with the possible implications of . 
vouchers. Many providers commented on 
the lengthy delays in receiving payment 
from the city and as such assume they can 
handle the variable payments from vouch- 
ers. Furthermore, many contractors placed 
such a significant emphasis on meeting 
their contracts’ enrollment goals, they 
believe they can manage with vouchers, 
even though the challenges under vouchers 
are different than they appear under con- 
tracts. As one director stated, “In theory, if 
we meet our. enrollment numbers under 
vouchers, we should be okay, right?” 
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Recommendations for Improving 
the Performance of New York City's 
Employment and Training System 



As a result of this assessment, we suggest a 
series of strategies to help, strengthen New 
York City’s employment and training system 
and its providers. We believe there are three 
broad areas that should be addressed to 
improve the performance of employment 
and training providers: systems level changes, 
organizational assistance, and near-term 
programmatic challenges posed by the 
Workforce Investment Act. 

Recommended Changes at the System Level 

Throughout the assessment, there were 
instances when the performance of providers 
was undermined by the functioning of the 
system overall, particularly in terms of per- 
formance measures and data collection.' To 
address these issues, P/PV offers several 
recommendations. 

Local performance standards should be aligned 
with national goals. 

City contracts should emphasize more 
strongly the same performance measures 
used by USDOL to judge the performance 
of the overall SDA, namely follow-up ^ 
employment and weekly earnings rates for 
adults and welfare recipients, and youth 
entered employment and employability 
enhancement rates. While these measures 
are included in the performance evaluation 
of providers’ contracts, diey are worth less 
than other measures that are not taken into 
consideration at all in calculating the overall 
SDA’s performance, such as training-related 
placements and average wage at emplo)Tnent. 
Given the emphasis training providers place 
.on meeting performance standards, focus- 
ing attention on the USDOL standards 
would likely have an immediate impact on 
tlie city’s overall performance. 



While WIA makes an attempt to coordinate 
performance measures with criteria to 
become an eligible provider, local boards 
should not add other measures that dilute 
the message to providers about what is 
most important in meeting the SDA’s goals. 

Performance information on individual contractors 
should be timely and accessible to.the field. 

While DOE has shared the overall SDA 
performance numbers with contractors, 
the key to improving performance will be 
sharing performance data at an individual 
organization level. Sharing the overall SDA 
performance numbers and the perfor- 
mance of individual contractors in a timely 
manner would focus attention on meeting 
key performance goals and on. developing 
clear strategies when goals are not met. 
Furthermore, release of this information 
will be required for the implementation of 
the Workforce Investment Act. 

The city should use performance information to 
weed out poor performers and shift resources 
to both stronger entities and to,new contractors. 

Given that the m^ority of providers have 
been in the field for more than 10 years 
and a DOE contractor for over seven years, 
a new process should be developed to offer 
opportunities to, other organizations to pro- 
vide] ob training services. As the analysis of 
individual contractor data indicates, the bot- 
tom quartile of providers achieved a median 
placement rate of only 50.1 percent. 
Creating a contract renewal system based 
on performance would provide. an opportu- 
nity to emphasize again to providers which 
performance standards are rnost important. 



A management information system .should 
meet the information needs of the providers. . 

Crucial'to enabling individual contractors 
to meet performance standards is their abil- 
ity to utilize performance information 
within their day-to-day operations. Creating 
a system that can provide both current and 
hikorical data on program operations 
would allow organizations to engage in 
more innovative program development: 

. Furthermore, it would greatly reduce the 
amount of time currently spent by staff to 
generate the same information, while pro- 
viding background information suitable for 
proposals to new funding sources and mar- 
keting to new participants. 



. The city should establish an independent Labor 
Market Institute. 

P/PV recommends the establishment of an 
independent, nonprofit organization whose 
sole responsibility would be to work to 
improve the performance outcomes of job 
training providers. .. 

The Institute, would need both public and 
private sector support in order to have the 
resources and flexibility required to meet - 
its objectives, and the operating needs'of 
employment training providers. ‘ 

The primary objectives of the Labor Market 
Institute are to: 



Whether tlie ACMS is continued or a new 
one developed, the system should be more 
user-friendly and tailored to the individual 
needs of providers. The system, while servr 
ihg the purposes of collecting data for the 
administering agency, should also be able 
to be manipulated by indmdual organiza- 
tions to meet their own needs, such as 
tracking their own trends in placement 
from year to year, arid to make strategic 
decisions about the future direction of 
the organization. 

Organizational Assistance 

Providers outlined a myriad of areas for 
technical assistance from "staff development 
to preparing for the Workforce Investment 
Act. While the technical assistance currently 
provided through DOE and the PIC 
received praise from several organizations, 
the variety of operating challenges and the 
depth of assistance that are required go 
beyond what the city agencies can provide. 
We believe that an independent entity will 
best meet these various needs and is far ; 
more likely to be able to leverage, private 
sector support from the foundation and 
employer communities. ' * 



7. Provide technical assistance and training 
on key^ challenges. 

Critical areas for technical assistance include: 

Staff development. Given the limited size > 
of providers’ staffs, it is imperative for 
organizations to attract and retain the most 
qualified employees. Providers recognized 
this challenge and overwhelmingly appealed 
for, staff development assistance, in both gen- 
^ eral terms and coveringjob specific duties. . 

In general areas, technology and communi- 
cation skills topped tlie list of requests. 
Equally important were Job-specific skills in 
outreach and marketing, Job development, , 
Job placement and postplacement retention. 
To meet these needs, the Labor Market, 
Institute would also serve as a. clearinghouse 
for information on professional develop- 
^ ment activities, as well as develop or obtain 
training curriculum for workforce develop- 
ment professionals. . 

To maximize these efforts, the.city should 
promote' connections to other professional 
development organizations in the field, as 
well as provide support to. each organiza- 
tion for professional development activities, 
such as purchasing materials or attending 
conferences. . 



Strategic planning. With rare exception, 
providers operate on a cycle-by-cycle basis 
with little opportunity for long-term plan- 
ning. Many providers have adopted a “wait 
and see” attitude and primarily, react to the 
city’s direction. Given the vast changes within 
the overall employment and training system, 
including the changes under WIA, providei*s 
need to engage their boards and staff in 
strategic planning in the following areas: 

• Financial management. With either 

performance-based contracting or vouchers, 
providers need to diversify their funding ' 
base and open or expand financiaf reserve 
funds. Reserve funds will be critical when 
training is conducted through vouchers 
instead of contracts. 



providers are granted additional funds. 
Both funds should be supported exclusively 
by foundations and employers. 

3. Analyze local labor market trends. . 

One crucial area for the Labor Market 
Institute would be the timely collection and 
dissemination of local labor market. infor- . 
mation. Since providers have been targeting 
the same industries for decades, sp ious 
consideration of other possibilities is over- 
due. This would encourage providers to 
focus training services on industries that 
offer the best opportunities for retention 
and advancement, while also keeping up to 
date with the evolving needs of employers. 








• Utilization of performance data in day-to-day 
operations. As discussed above, boards and 
staffs need to understand what information 
about themselves is crucial to enticing new 
funders and participants. In addition, serious 
consideration needs to be given to investing 
in a technological infrastructure to make the 
changes work. 

• Partnering with other organizations. Given 

movement toward larger organizations . 

subcontracting with smaller organizations, 

' . carefulxdnsideratibn needs to be given to , 
finding, and building, the most advantageous 
financial and programmatic partnerships. 

2. Create a loan fund for providers to access to 
meet short-terrfi cash flow problems. 

Many current providers’ financial situations . 
are weak. Most struggle to meet their. basic 
cash flow requirements, since they have lit- 
tle, if any, in the way of financial reserves. 
While some limited support is currently 
available, the emergence of performance- 
based contracts and a voucher system 
requires a more significant investment in 
this area. The Labor Market Institute could 
also develop a risk capital pool that could 
be used to support promising new strate- 
gies. It would also be responsible for 
developing guidelines for applications and 
establishing policies regarding the amount 
of the loan and the frequency with which 



4. Serve as a forum to build knowledge about 
effective piractice and policy. 

As discussed, pro\dders have little access to 
information regarding developments in the 
workforce development field, both locally 
and around the country. Therefore, the 
Labor Market Institute could serve as a con- 
vener of forums and conferences to connect 
providers to knowledge being developed 
throughout the country about what works. 

Near-Term Programmatic Changes Under 
the Workforce Investment Act 

Many of the reconimendations noted above 
will take time to implement. The challenges 
posed by .WIA cannot wait, however, and the 
city and its providers need to begin address- 
ing therri now. Aside from management 
issues, changes, particularly related to WIA, 
will force providers to examine their pro- 
grammatic structures and possibly make 
adjustments. Organizations will need assis- 
tance in restructuring their programs, if 
they are to perform well under the new leg- 
islation. Among the many changes are the • 
need for: 
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• Multiple points of entry. Under the current 
system, participants have basically two . 
opportunities per year to enroll in training 
programs. Under a voucher system, the 
number of entry pofnts into training 
programs would be expanded to allow 
potential participants to use their vouchers 
close to the time they are issued, not when 
programs are recruiting. These multiple entry 
points will allow organizations to maximize 
opportunities to achieve full enrollments. 

This changed structure rhay have serious 
implications for providers’ operations. 

• • Expanding postplacement services. 

Increasing the follow-up period for 
performance measures from three to six 
months may prove to be a very difficult,^ 
challenge for providers. Few providers have, 
dedicated. staff to providing retention 
services and, those that do, focus more on 
employment verification than actual - ■ 
posLplacement assistance. Given the . 
heightened importance ofjob retention 
under WIA, providers will need help in 
developing and implementing concrete ^ 
posLplacement services. . ' 

® Collaboration with the one-stop operators. 

The Vctst majority of providers still rely on their 
own recruitment methods to meet enrollment 
goals. Shifting to a centralized system will 
require some programmatic changes. The 
providers currently-involved in the city’s pilots 
of centralized referral and assessment centers 
insist that one^n-one relationships with one- 
stop operators is the key to their success. In 
designating a liaison with one-stops, providers 
also need to clearly define their programs and 
the type of participants they require. 





Conclusion 

New York City’s employment and training 
community faces a multitude of changes 
and challenges over the next few years. 

A variety of issues, from performance 
standards and financial constraints to the 
impact of welfare reform and the new 
Workforce Investment Act, need to be 
addressed to improve overall performance. 

It was not always easy to engage some of the 
providers in a discussion about technical 
assistance heeds and how to improve the 
overall system. Many doubted whether the 
necessary resources would be put forth to , 
implement the recommendations. Others 
felt that a new technical assistance program 
would be more beneficial when further work 
has been completed defining the parame- 
ters of the Workforce Investment Act. 

However, we believe that investments ^ 
should be made now that will strengthen 
providers’ ability to meet the needs of 
employers and pa.rticipants. 
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Agudath Israel of America, Inc. 


LaGuardia Community College- 


America Works of New York, Inc. 

■ . . ■ 

Argus Community, Inc. . 


Metropolitan New York Coordinating Council of 
' Jewish Poverty 


. Binding Together, Inc. . - 


Multitasking Systems of New York, Inc. 


Bronx ‘Community College 


New York Foundation for Senior Citizens, Inc. 


Center for Employment Training 


New York Urban League 


Center for Employment Opportunities, Inc. 


New York City Technical College 


Chinatown Manpower Project, 


Nontraditional Employment for Women 


Chinese-American Planning Council * 


Northern Manhattan Improvement Corporation 


Community Associates Development Cprporation . 
Consortium for Worker Education 


New York City Health and Hospitals 
. Corporation, Kings County Hospital 
Center/Educational Vocational 
Rehabilitation Program 


Council for Urban Employment, Inc. 

Council of Jewish Organizations of Flatbush'^ 


New York University Hospital Center - . . 

Rusk Institute ^ 


East Harlem Council for Community 


Opportunities for a Better Tomorrow 


Improvement, Inc. 


South Forty Corporation . 


Ecumenical Community Development 
Corporation / 


Staten Island Employrhent Education Consortium 


Elmcor Youth and Adult Activities, Inc. 


The College of Staten. Island . 


Good Shepard Services ' 


Wildcat Service Corporation 


Hellenic American Neighborhood Action 
Committee, Inc. 

International Sheet Metal Workers/Iron Workers 

Jewish Community Council of Greater 
Coney Island . 


Xincon Technology School 
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Endnotes 



1 Five of these organizations run long-term 
training programs as well as TAP Centers. 



9 Attachment A lists the 37 providers who 
responded to. the survey. ' 



2 Service Delivery Areas are designated by the 
governor to receive federaljob training 

, funds. Local governments with populations 
of 200,000 or more are automatically eligible. 

3 Follow-up employment rate is measured on 
the 90th day after termination from the 

- . program. ‘ 

4 Follow-up weekly earnings are also measured 
'90 days following program termination.' 

^ 5 Overalbstate performance numbers are an 
average of all the individual SDAs. 

6 Contractual standards are' based on . 

additional USpOL benchmarks, known as 

' noncore performance'mea^ires. Prior to 
1990, these measures, which included 
placement at termination and average wage 
at. placement, were the indicators used to 
judge the overall performance of the Service* 

, Delivery Area. .. . . * 

7 New York state utilized the maximum 
adjustment possible in most oLits standards 
for New York City. 

“■ t- 

8 Eleven contractors run both Title IIA and 
Title III programs. 



10 Twelve of the 19 organizations that have c^h 

reserves reported a specific amount. ' ' . 

11 In addition, the Professional Re-employment 
^and Outplacement Services (PROS) has its 
own center to meet the needs of more skilled ^ 

' displaced workers. 

12 Initially known as Assessment Centers. 



■ 13 NCDCs also serye JTPA youth providers in 
. ' the Bronx, v 

14 Some organizations that ran multiple 
programs did have shorter cycles, such as 
eight-week programs, with four entry points 
per year; but these were usually for 
specialized populations. 

15 Submitted two surveys from two different 

training programs. ' . 
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